SCORE: approximating curvature information under self-concordant regularization

Abstract Optimization problems that include regularization functions in their objectives are regularly solved many applications. When one seeks second-order methods for such problems, it may be desirable to exploit specific properties of some these when accounting curvature information the solution steps speed up convergence. In this paper, we propose SCORE (self-concordant regularization) framework unconstrained minimization which incorporates Newton-decrement convex optimization. We generalized Gauss–Newton with Self-Concordant Regularization (GGN-SCORE) algorithm updates variables each time receives a new input batch. The proposed exploits structure Hessian matrix, thereby reducing computational overhead. GGN-SCORE demonstrates how convergence while also improving model generalization involve regularized under framework. Numerical experiments show efficiency our method and its fast convergence, compare favorably against baseline first-order quasi-Newton methods. Additional involving non-convex (overparameterized) neural network training is promising